1. Field of the Invention
The present invention relates to search engines and database searching techniques and more particularly pertains to a new method and system for automatic harvesting and qualification of dynamic database content for efficiently providing highly relevant and timely information in response to user's queries.
2. Description of the Prior Art
Many enterprises, whether business, governmental, or any other organized undertaking, require large amounts of “current” information to be analyzed and available for use in the daily execution of their activities. Often the informational needs of the enterprise can be categorized into discrete subject areas or domains. Each of these domains may have additional divisions providing increasing granularity or specificity of the subject matter.
Since its inception, the Internet has held the promise of real-time access to an almost inexhaustible supply of information, stored on computers throughout the world, in near real time. However, sorting through the information available to find documents relevant to a given question or query can be laborious; and a method to speed this process was needed. Search engines are known in the prior art and allow a user to search for sites that have some keyword corresponding to the user's query. While it is true that millions of documents are readily available as static pages to users through search engines, much more of the total content of the Internet, in the form of dynamic content, has remained relatively difficult to access through more conventional search engine techniques. For the purpose of clarity, a static page of a network database provides the same content to virtually every user accessing the database, usually in the form of the same document or page (or set of documents or pages). A dynamic network database presents dynamic content to each user accessing the database, and the dynamic content usually comprises unique documents or pages that are in response to and are based at least in part on the user's query.
The dynamic content, while available, often requires independent knowledge of the exact location of the document, sophisticated search techniques, or in many cases the use of professional researchers to attempt to “mine” the needed information.
Additionally, the resources required to evaluate all of the information identified by a conventional search engine in order to filter out non-relevant information can be more than substantial. The resources used may include, by way of example and not limitation, transmission bandwidth, data storage, and time (both of system usage and of personnel) required to filter out related but not relevant information. The need to capture and organize relevant information can be overwhelming, and an automated system is required to effectively solve this problem.
In these respects, the method and system for automatic harvesting and qualification of dynamic database content according to the present invention substantially departs from the conventional concepts and designs of the prior art, and in so doing provides a system primarily developed for the purpose of efficiently providing highly relevant and timely information in response to user's queries.